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ABSTRACT 

The primary goal of the i^tfisconsin Reading Task Force 
is the initiation and implementation of an immediate and ongoing 
program of statewide assessment of reading performance and reading 
instruction. The assessment plan looks at students in terms of two 
questions: How well is the student achieving in reading in comparison 
with other students in his class^ his grade^ and at his age across 
the city and/or nation? Can the student read the material that the 
school and society expect him to read? The first question was 
answered by looking at the standardized tests administered each year. 
An^ analysis of student achievement on tnese tests showed that Madison 
students do on the average read as well as or better th:in students 
from cities across the nation. The second question has not been fully 
assessed, and it is this question that the assessment plan is 
attempting to answer. Students focused upon in this plan were fourth, 
seventh, tenth, and twelfth graders. Reading was defined as an 
ability to gain information from what was read. The cloze test was 
used as the measuring instrument; this procedure was selected ibecause 
of its utility, validity, and objectivity. The final report of this 
project is due to appear shortly. (WR) 
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PART I 
PRELIMINARY COrWENTS 

A Brief Historical Review 

In 1969 President Nixon's Administ.ntion declared that the 
most important target for the decade ahead was the achievement of the 
"right to read" goal of literacy for all citizens. 

In the spring of 1970, the Department of Public Instruction, 
IB response, prepared a position paper entitled "The "Right to Read*: 
a Wisconsin Plan." The plan set as the primary goal of the Department 
of Public Instruction Reading Task Force the following: To pn,ovidt 
CAACJumtmcu duignzd to imtilZ tht ^hUZ and duOit nzcu^a^ to 
KQjod wWUn the. ^uJU turuX6 o^ kU aipacity iox tvoAy indivMiat in 
ou/i 60(UeXy. 

Within the context of this primary goSl, the Reading Task 

Force had five action goals, one of which was... ^» 

"The initiation and implementation of an immediate 
and ongoing program of statewide assessment of 
re'^Jing performance and reading instruction within 
the State of Wi scons. n." 

To this end an assessment committee was charged with the 
task of developing a plan for assessment of reading literacy. In 
March of 1971, Dr. John Gottman, a staff member at the University of 
Wisconsin Instructional Research Laboratory, submitted such a plan to 
the Department of Public Instruction. 

It is this plan that the Madison Public Schools, in coopera- 
tion with the Instructional Research Laboratory and Department of 
Public Instruction, is piloting. 
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The Scope of the Assessment Plan 

An assessment of a reading program could possibly examine 
that reading program from a number of points of view. It could look 
at teacher background and expertise. It could look at teacher 
behavior in the classroom. It could look at teacher-education (pre- 
service and inservlce) programs. It could look at the books and 
materials used In a reading program. And finally it could look at the 
student to see how well he is reading. 

The assessment plan presented here chose to look at the 
student. When looking at the student these two questions come to mind. 
1) How well is the student achieving In reading in comparison with 
other students in his class, his grade and at his age across the city 
and/or nation? 2) Can the student read the material that the school 
and society expect him to read? 

The answer to the first question can be found by looking at 
the standardized reading tests typically administered each year. In 
Madison we administer a number of such tests. For Instance, this year 
we are using the following: 

- Grade 1. The Clymer-Barrett Reading Readiness Test 

- Grade 2. The Gates MacGinitle Reading Test 

- Grade 3. The Gates-MacGlnitle Reading Test 

- Grade 4. The STEP Reading Test 

- Grade 5. The STEP Reading Test 

- Grade 8. The STEP Reading Test 

Each of these tests with the exception of the readiness test 
typically contain a vocabulary section and a comprehension section. In 
the comprehension section students read a number of prepared paragraphs 
and then respond to carefully constructed multiple choice questions. 



An analysis of student achievement on these tests shows that 
Madison students do on the average read as well as or better than 
students from other cities across the nation. This might lead members 
of the comnunity to be satisfied with the instruction children are re- 
ceiving in school. When looking at student groups within the city, the 
analysis reveals differences amang groups of students. This naturally 
raises questions regarding factors that might contribute to the differences. 

When carefully thought out questions are raised, answers should 
be systematically prepared. These questions would likely fall in the areas 
of the nature of instruction, the equality of instruction, and differences 
In social economic backgrounds of students. In the main, it appears that 
the Madison Board of Education, through the schools, materials, teachers 
and support personnel, is providing children with quality instructional 
experiences. Once again though, it should be noted that the answer to the 
question "Do Madison students achieve in reading as well as other students 
across the nation?" is yes. 

The second question "Can the student read the material that the 
school and society expect him to read?" is quite different from the first. 
A con^letely different type of Information is required before this question 
can be answered. It is this question that the Reading Assessment plan 
attempts to answer . Therefore, the rest of this paper deals with how the 
Madison Public Schools in cooperation with tho Wisconsin Department of 
Public Instruction and the University of Wisconsin Instructional Research 
Laboratory set about to answer this question. 
The Nature of the Assessment Plan 

To introduce the nature of the assessment plan used in Madison, 
a number of clarifying questions about "Can students read the mdterial 
that school and society expect them to read?" are raised and answered. 



Question: Is this question usually asked and answered? 

Answer: Asked, 3'es. Answered--wel 1 , teachers sensitive 
to reading know part of the answer, but actual 
data is not systematically gathered. Because 
of this, Mad-'-on is engaged in a unique assess- 
ment. 

Question: Who are "the students"? 

Answer: Students focused upon in this plan were fourth 
graders, seventh graders, tenth graders and 
twelfth graders. 

Question: What does "read" mean? 

Answer: - "Read" sometimes means "to be able to associate 

the correct sound with the given letters in words-" 
By listening to a child, another person could tell 
if he were "sounding out" the words in the message. 
- "Read" can mean "to be able to comprehend or under- 
stand what is written." This definition approaches 
what we mean here by "read." The problem becomes 
what is meant by comprehend. 
• So more specifically, "read" for us was defined as 
"the ability to gain information from what was read." 

Question: What is meant by "material?" 

Answer: Anything printed that schools and society give to, 
address to or suggest to students at a particular 
age. For instance, society seems to expect 12th 
graders to "read" financial contracts; the publishers 
of the magazine Bo y's Life expect 7th graders to be 
able to "read" Boy*s life . 



Question: How do you measure whether or not students can 
"read" (gain information) from a chosen piece 
of material? 

Answer: - One way is to talk to the student and find out 
what information he gained from the message. Of 
course, this is impractical and may prove to be 
unreliable (too rmjch left to interpretation on the 
part of the interviewer). 

- Another way is to carefully construct multiple 
choice questions as the standardized test people 
do. This would be quite an undertaking when you 
consider the number of different materials you 
would like to find out if students could read. 

- A third method of measuring whether or not students 
are gaining information is the cloze test of 
reading comprehension. 

This reading assessment uses the cloze test . 
End of Questions? - 



Vou havz cut tkU point jvudt ilYiUh^ xexidiyis ""ha p^eJUmim/iy 
comrnnt^. 1(5 m havo. mAXtzn i^aZl, you shoaixi dc able, to an6i^(2A 
i/ieAe gonoAol quution^. (/) Who initiated tha MiadLi\/\Q (U6U6- 
mmt? (2) What pnAjm/iy quution ^ the, xzading a^^e^^tmnt 
asking? (3) What othek que^tlon^ about Kejoding ojie. a^ked ayid 
hoijo axt tkeJ^z an^voexexL ll not by tki6 a^^iU^rmnt? [4] In the, 
a&6e^Amejit, voho a66e^6e,d, u)hat l6 a^^e^^ed, and hou) iM it 
a&4>e^6ed? 



I^' you axz unj^uAZ o^J thz aYU^vooM, jot dom quejbtionj^ that you 
maid likz to a^k. 



(J) WluLt Is tht clozz tut? (2) Why nbc it? 
(3) Hoijc dcu the. dozz tut KcJUtz to LiteAacij? 
[4] How was thz as^u^rmnt JbeX up? (5) What ^ 
qautxons should tiiz as^u^mdnt ^aise? 

PART II 
WHAT IS THE "CLOZE TEST"? 

To explain what the cloze test or procedure is can perhaps 
best be accomplished by describing the procedures for using it. 
Bormuth (1968a) set forth five succinct steps: a) a passage is selected 
from given materials, b) every fifth word in the passage is deleted and 
replaced by underlined blanks of equal length, c) the passage (test) is 
then duplicated and given, without time limits, to students who have not 
read the material from which the passage was taken, d) students are 
instructed to write in each blank the word they think was deleted, 
e) responses are scored correct when they exactly match (disregarding 
minor misspellings) the words deleted. According to Bormuth, " When the 
tests have been made properly, a student's score can be interpreted as 
how well he understands the materials from which the tests were made ," 

On the following page you will find a sample cloze test. This 
passage was taken from a pamphlet on bicycle safety and is published by 
the State of Wisconsin Division of Motor Vehicles and is distributed to 
young bicyclists. 



RULES FOR BIKE DRIVERS 



Drive close to the right side of the 



single file, and pass cars with cere. 



Sit the bicycle seat when , % 



and never carry extra — no "trick" riding. 



Never rides on other vehicles. 



carry loads which prevent 



from keeping at least hand on the handlebars 



all times. (Or better, attach a basket to 



your biko for packages, so you always have both hands free 



for driving. ) 
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The correct answers to this cloze test are 1. road 2. parked 
3. on 4. riding 5. passengers 6. hitch 7. never 8. you 9. one 
10. at 

The cloze test since its conception (Taylor, 1953) has been 
used in two ways. First, it has been used as a readability procedure 
to determine the appropriateness of instructional materials for children. 
And second, it has been used as a measure of students' ability to compre- 
hend (gain knowledge from) materials. The two uses are not really differ- 
ent for the cloze procedure in determining the appropriateness (read- 
ability) of material does so by measuring if the student can understand 
the material. The difference in the two uses is simply the focus — the 
material or the students' performance. 

The cloze procedure is not a "half-baked" technique. A recent 
comprehensive bibliography on the cloze (Klare, 1971) lists 159 articles 
and/or research reports on the use of cloze. A check of the titles 
suggests that 39 of the entries deal with cloze as a readability measure 
and 105 discuss the cloze and its relation to comprehension. One of the 
foremost experts on the cloze, author of 20 of these entries, is Professor 
John Bormuth of the University of Chicago. Dr. Bormuth is serving as a 
consultant to this assessment project. 

PART III 
WHY USE THE CLOZE? 
Reasons for using the cloze procedure in the Madison assess- 
ment are discussed below under these three headings: utility, validity 
and objectivity. 
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Utility of the Cloze Procedure 

In order to answer the question "Can students read the material 
that the school and society expect him to read?" it was necessary to test 
students on a wide variety of materials. (For a complete listing of the 
10 domains and 60 categories, see p. 16 in this paper and Appendix A, 
p. 21.) The cloze procedure can be used on any material containing 
more than eleven words. Consequently, once the materials of concern are 
identified, it is quite easy to turn them into a test by using the cloze 
procedure. This allows the test constructor to forego the lengthy and 
arduous task of constructing questions and establishing their reliability 
and validity before the test is ready to be used. 

Another utility feature of the cloze procedure has been sug- 
gested earlier. This has to do with what might be called the "double 
duty" aspect of the cloze test. As you recall, results on the cTbze 
test can- be looked at as saying something about both the students' 
ability and the material. 

Validity of ths Cloze Procedure 

When asking if the cloze procedure is valid, one is in essence 
asking "Does the cloze test in fact do what it purports to do— namely 
(1) measure the comprehension difficulty of the material, and (2) measure 
the comprehension ability of the student? To answer this question, the 
validity of the cloze procedure is discussed under the sub-headings of 
(a) logical validity and (b) concurrent validity. 

Loc^ical validity 

In establishing logical validity,or face validity as it is also 
referred to, the target (the cloze procedure) is focused on through re- 
flective glasses. In fairness to this reflective process, go back to 
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the one cloze test included in this paper (p. 7) and think about it 
and what you did as you were "guessing the missing word." Hold these 
thoughts in your mind as you read below some comments Bormuth (196Sa) 
has made about the logical v^'^'^i ^ the cloze procedure. 

cZozz KzadabUAXy te^t appzjOM to be a complatzty diiiojizwt 
kind 0(J toAit and ^omz autkou havz made much o|J tlii6 lact, 
attmpting to coyUitAuct all C|J my^ticai^ tkzonldb about 

diozz te^t^. Some havz p^oie^6zd to 6zz a bimiloAJXy fae^ftveeu 
tkz pK0CQ/i6Q/i involved in ^e^i^ponding to a diozz toAt and tkz 
ctozuAt phznomznon ob^QAvzd in thz poAczption OjJ gzomatyiic 
{^iguA-U. Indzzd, it u;aA i^om ju/^t thiM kind oi conjcctuAz 
tkaJt cZozz pKoczduAz got it6 name. 

"On cZc62A in^pzction it can be 4een that many o^ tiiz 
itm& in ciozz KexidabitUy te^tb oAe idznticat to tlio^z 
lound in reading compAzhzn^^ion tz^t^ madz by convznZionaZ 
mzMiod^ and that thz pAoczM^zM AzquiAzd to iiZl ctozz blanks 
oAz pAobably not diUzAzwt iAom tho6Z AzquiAzd to ar^tceA 
convzntionalZy madz itzm* 

"Convzntional complzXion tzA^t item oAz madz by simply 
dzJizting a mnd oa phAo^z f^Aom a szntzncz. ¥oa zxamplz, 
givzn thz szntzncz ''Thz boys Aodz hoASz^," it i^ possiblz 

to makz thz complzJUon quz/^tions, Aodz hoASz^, Thz boys 

, Thz boys hoASz/^, and Thz boys Aodz . Thz 

{^amJUoA yoh- questions oAz madz in much thz samz mannzA. 
A vooAd OA phAo^z iA dzl(itzd, a voh- phAo^z i6 in&zAtzd in its 
placz, and thz szntzncz iJ> tAansf^omzd so tliat it bzgini^ mth 
thz ijoh- phAo^z. TkU givz/i thz questions. "Who Aodz houzA? 
What did thz boys do? What did thz boys do to thz houzA? 
and What did thz boys nidz?" As in clozz tz^sts, thz conAzcX 
ansvozAS to thzMz quzA^tion^ oAz thz ijooAds oA phAo^zh dzlztcd. 

" ContAosts mXh Convzntional TzAts * But itms madz by 
cioiz and convzntional tzAt ma\iLng pAoczdvJizM d^UzA in 
szvzAol inpontant Az^pzcZs^. ¥iut, in a diozz Azadability 
tz^t, only onz mAd i& dzl^zizd at a timz i^hilz in convzn- 
tiomZ tz^ts, voholz phAosu and cJLolujs^zm may also bz dzlztzd. 

"k szcond majoA diif^zAzncz is thz iact that clozz 
Azadability tzA^ts oAz madz only {^Aom thz szntzncz^ in thz 
tzxt i/o^iilz convzntional tut items may bz madz cithzA (^nom 
szntznczA in thz tzxt oa f^nom thz szntzncx*^ that can be 
doAtvzd {^Aom thz tzxt. 
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"AnotheA majo^ cont/ui&t thz jjact tlxat ctozz 
KzadabitUy tut6 axz txikm by ^tudznts u)ho havz not Kzad 
thz undztttQjd vzuion ojj tkz pcU6agz. 

"P/Lobably too much h(i6 bzzn madz OjJ thuz cont/uuts 
between cZozz and convzntLonal te^tb. Thz 6tudznt ha^ 
ziqhty pzAcznt OjJ thz text on lokich to ba6z kU ^Upon^tb, 
60 hi6 KUpon&u vzAy mixcJi dzpznd on kU ahitity to andoA- 
6tand thz tzxt. Mbo, thz jjact that hz ha^ not ^zad thz 
oKiginat tzxt may ^zquAJiz that hz ii^z p^ocz66U 6imWvi 
to tko6Z KzquJjiQxL to an^voan. quutlom madz ^om dvuvzjd 
6^ntzncu piu6 a 6Zn6AXivity to thz author* 6 6tylz and 
thz tonz ojj thz pa66agz. HovozvoA, thz cont/uut6 do zxJjbt 
and 60 thz quQ>6tion mii6t bz ^z^OA/ied to thz ^UOjOAchoA, 
thz iinat oAbitoA ojj 4ucfi dl6putt6." 



Bormuth in the comments above has chosen to establish the 
logical validity of the cloze procedure by comparing and contrasting 
the cloze procedure with conventional tests. In so doing, Bormuth 
points out that while the cloze procedure appears to be different, 
the correct answers for the cloze are frequently the identical answers 
required by a more conventional test question. In addition, Bormuth 
suggests that in order for the student to respond with the correct word 
In a cloze blank, he needs: 

1. to understand the meaning of the words that are used. 

2. to attend to the relationships that exist within the 
sentence, and 

3. to respond to the author's tone and style. 

This process above does, in fact, seem to be the process a 
reader goes through as he takes a cloze test. This also seems to be a 
logical description of what a reader has to do in an effort to comprehend 
(gain Information) from a message. 
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Concurrent validity 

When turning to concurrent validity, the reflective process 
involved in establishing logical validity is put aside. In its place 
the researcher, or final arbiter as Bormuth labels him, with his tools 
and skills assesses the topic. Here is a sunmary of what some research- 
ers have found in answer to the two questions asked in regard to the 
cloze procedure. 

1. Does the cloze test procedure measure the comprehension 

difficulty of the material? Bormuth {1968a) reports: 

"Tfie mean poAcznt 0(J item a g/uoup avu^vozn/^ cofUizcXJbj on 
a clozz ^tadabiZity t(Lbt AeemA to pKovidz an accxxAatz measu/iz 
0^ tkz (UiiicuJUy o(J tho. p(U6agz, aJbno6t /iega/idlU6 OjJ hou) 
dliiicjuJUbj lb tma^uAzd. Shiba (1957) ^ound a coxxelation o^ 
.83 be^een t'lo. diazz /lejodabilitiu o(J poA^ogeA and thz combimd 
6ubje.ctive. njuuting^ o^ the. pa^6age^ by tJtOizz judQejii. Bomath [1966] 
^ound a co^eZation 0(J .92 be-'/iceen tke. cloze. Kojxdabititie^ oi 
pcbb^aqeM 06 mexu>ixAzd by mattipte. choice. teJ>tSi. Subsequently, 
Bomath {196Sb) u6ed each o^ the. iouA {^om6 o^ the. Gnjay On,al 
ReadinQ PaAagnjapkb. He. iound conjiclations^ nanQinq in.om .90 to 
.95 beJbJOce.n the cloze and voond /lecognition di^^icultie^ o|J the 
paAagnapk6 and con/teZatiom^ n,anging i^om .91 to .98 between the 
cloze and the compn.ekeyi&ion difficulties of the poAog/iaptu." 

2. Does the cloze test procedure measure the comprehension 

ability of the students? Bormuth (1969b) reports: 

"NeoAXr/ all of the n,e&eaAch has shouon that 6cc^e& on cloze 
te&ts a/ie kLghly conxeZated voiXh scon.eJi on standardized te&ts 
n,eading compn.ehension abiJUty. Ton. example, Jenkinson (/957), 
RuddeZZ {1963), and Bonmuth {1965) found conAeZations i^kich gen- 
enxULZy n^anged fn.om .70 to about .S5. 
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Objectivity of the Cloze Procedure 

The third reason for using the cloze procedure is its objectivity. 
The construction of the cloze test is very straight forward. Once materials 
are selected, explicit rules for deleting words are followed. Actually the 
rules for selecting the materials themselves are explicit once a type of 
material has been decided upon. 
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Scoring of the cloze test, while time consuming, is quite 
objective when the correct response is the actual word that has been 
deleted. The only judgments scorers need to make are related to 
spelling. Then, if the word is recognizable, it is accepted. 

An additional feature of objectivity is that is is difficult 
if not nearly impossible to "teach for the test." The test is not a 
knowledge test. The test samples an extremely wide body of actual 
passages. If the student had been given practice working the cloze pro- 
cedure and was able to do well on the cloze test—great! After all, we 
want students to have the skills needed to respond well on the cloze— 
if they have those skills, they can read! (During the test 
situation, students are given a sample cloze test to "warm-up" on.) 



PART IV 

HOW DOES THE CLOZE TEST RELATE TO LITERACY? 



What Is Literacy? 



Bormuth (1970, pp. 1-2), in the three paragraphs below, a - 

cusses the importance of literacy and what literacy is and is. not. 

"Mo6t pojoplt ag/izz that tht mttmcutt pixxpo^z 0(J K^ding 
^mt/iuction to mabtz 6tujdznt& to undc^utand i^hat the,y n.eM. 
WAMiout tkz lltviacy 6kiU^ It l6 mpo&^lblz ^on, a ptuon to 
cZoAjm kit bi^th/Ught oA an AmeAJ.can; in 6ckool and thAoughout 
hit -tcjje a pmon mu6t n,eJbj mpon kU UtoAacy 6kilt6 in 
cJuJxcai ijoay6. IjJ a6 a student hz i6 unablz to undoA^tand Ivu 
;textfaoofe6, kz i& abno6t coAtainty deAtinod ^on. {^aiiix/iz aiful 
QjOAly dAop-out, making iZ di^^icutt ioK him to obtain a 6tmiy 
and dQMinRblz job. Th^ca^toA, it i& di^icutt ioA ki/n tv 
6tudy thz mcitQAialSi that could pAovidd turn u)ith thu UchucaZ 
t/uUyUng nzc2^6aAy to i(r\pH.ove. kiJt> zmptoy ability. He i^ ^H^c,' 
tXvoXy diMinf^Kanciuzdd by ki& cuhXa^i^id ability to iifif^um kirn- 
aqZ^ on political ^6a^>6. And he >6a dcmQd accc'^A the nick 
cuLCuAol htnAJtagc mbodied in oua litMatuAe.. In iihunt, ka u 
fiOACdd to (xviL on the maAgint^ o^ oixn ^ocJd^ty. 
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"A decade tvoo ago rmch tiiz pmbiic and mamj odaccitu^xt 
adopts an ovqa- 6 Jjnp tidied viw t/ie naXuAC and caa<se c';^ t/ic 
iZti-^.QAacy pu^blm6 In tlu^ caantAij, a v^m that laid mcit 
tkz biamz on thz 6ckooW iaiZuAe^ to p^ovidz zailij and t^ijbtan- 
atic >Ln^tmction in pkonich. Thz Aca^oning bdilnd tliU position 
u;aA i<uAJbj 6ound in 6omz Kii^pdjdXA^: it u;aA argued that a poA^on 
voho can/iot at tzoj^t idmti^y thz t^XfAd& on a j^age cannot pouibltj 
undzutund uohaX tho6Z i/ooKds^ mean. FuAthznmoKe, theAZ tva^ ecu- 
6ideAabZ\\ ju&tiiicxuUon at that tanz ioK thz choAgz that pkcnic6 
in6tMiction in thz 6ckooJU w;aA nzithoji 6y6t£matic no a voAy 
zi{^zctivz. Homvz^, tku aJbno6t ZKciu^ivz attzntion givzn to 
mcu^tvuj oi tiiz 6ifnplz 6kiUU that znabtz a child to call, oA 
pAonouncz, thz mnsh hz 6zqm in pKint has^ Iz^t mo6t thz 
pabtixL and many zducatonJi mth a dangz^ouAly yuiAAouxid conczpt 
that zquate^ a child' 6 ability to call mnd^ u)ith literacy. 

"In point o(J (Jact thz ability to call mKd^ idim takzn by 
it&zli AzpKZ&znt^ a 6zt ofs 6kiW> 0(J Kzlativzly titxlz mpoAtancz 
Ton. zxamplz, mo6t AzadoA^ can pKowouncz thz non6zn6Z mnxU in thz 
6zntzncz Thz Vaxu mn^gzd thz bo^uxU in thz toaygap . Bat ccxr 
tainly no onz maid clahn that hz dvUvzxl any social, zcjonomic, 
on. intcJUzcMxal bznz^^U in.om doing 6o, because hz obtained no 
inionmation in.om thz 6zntzncz. Hence thz vaMiz u)z dznlvz ^nom 
Kzading dozA not come in.om thz mz^z pAomincxation o^ mnd^ bat 
KothoA inx>m thz inionmation tee obtain by 6ab6zqazntly compnz- 
hznding thz in^onmation n.zpn.zi>zntzd by tho6Z mnxU. When u)z 
6pzak 0|J tUznacy, then, u;e oaz n.zatly KzicMing to a complzx 
act con6i6ting o^ both voond Kzcognition and thz compAzhen^i^ion 
that iollom. Any noAXoiJOOA conczption o(J litznacy is both 
trivial and afa^uAd." 

In summary, Bormuth associates literacy with the ability to 
.ead. For Bormuth the ability to read is synonymous with the ability 
to gain information- The process of gaining information is viewed 
as a complex act consisting of both word recognition and comprehension 

The question we are asking in the reading assessment is 
"Can students gain information from the materials that society and 
school expect them to be able to read?" 



Determining Literacy 



One way of measuring whether or not students are gaining 
information would be to give them a multiple choice test before they 
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read a passage; then let them read the passage; and then give them 
the multiple choice test again. By subtracting the number correct 
on the first test from the second test you have an index of the 
Information gained. 

In essence, Bormuth (1969a) did what was just described. 
But, he took the procedure one step further. Using the cloze procedure, 
he tested students to determine their cloze test scores on the passages. 
Jhen by using appropriate statistical procedures (a regression model) 
which related the cloze score to the multiple choice test Jndex 
of information gain, Bormuth was able to turn the cloze score itself 
into a comparable Index of Information gain. 

It appears (Bormuth, 1969a, 1971) that when a student 
scores about 35% correct on the cloze test, he is gaining little or 
no information from the passage. Students, then, who score 35% on 
the cloze test over given material are at best only minimally literate 
with respect to that material * 

This reading assessment is asking whether or not students 
are minimally literate with respect to the material they are expected 
to read. 

The. ldz<u^ ^uantad in tka la^t {^m paAjoLQnwph^ 
Kzpiuzvvi a xexkiction an exieiaA>ci;e body oi n,(i- 
^eo/LC^. kt but thz p^umtcuUon It ^miptiMtic. 
Hapz^uJUy, you voUZ n,ejOid and XQAojxd tkoJ^a pfviagnxiplu 
uuXk tkl6 in mind. quutiont p2A^i6t, jot tixm 
dom, a^k ^oK cloAi^iaation, go to tkt K^oJimcu 
tUted, dtc. 
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PART V 

THE DESIGN OF THE READING ASSESSMENT STUDY 
What Materials Were Actually Used? 

If we are assessing the literacy of students with respect 
to certain materials, a reasonable question Is "What are the materials?" 

Appendix A contains a complete list of materials by domain 
and category. 

Bejjo/ie you n,ejcul on, take, a look at Appendix A. 

In Appendix A after each category, the grade level at which 
these kinds of materials were used Is Indicated. Also, If the same 
representative sample from a category of material was used at more 
than one grade level. It Is Indicated. For Instance, In the domain 
Reference Materials the passages used within the category "Telephone 
directory" were the same at all four grade levels. On the other 
hand, the passages from the category "Encyclopedia" differed at 
grades 4, 7, and 10 because students use or are provided with different 
encyclopedias at the different grade levels. 

Who Determined What Materials Should Be Used? 

A brainstorming process that took place over tL number of 
days and Involving a number of people was used to develop the list 
of materials represented by the sixty categories. The people In- 
volved Included thirty-nine teachers and administrators (many of 
whom were parents) from Madison and neighboring conmunlties. 
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Once the categories were established the actual gathering 
of sample messages was done in a systematic and replicable way. 
For instance, sample messages in the category "Promotional literature 
on new cars" were gathered by soliciting new car dealers chosen at 
random. Sample messages in the category "Newspapers" were gathered 
by randomly and systematically sampling the lead news stories, edi- 
torials, feature stories, political features in the Milwaukee Journal . 

Materials were assigned as "appropriate and expected to 
read at grade 4, etc." by one of two methods. One , the material 
assigned itself. Textbooks purchased for fourth graders are for 
fourth graders. Recreational department bulletins distributed to 
fourth graders are for fourth graders, etc. Two^ for those materials 
whose grade level placement was not clear cut, sewing directions, 
newspapers, consumer magazines, etc., placement was based on the 
results of a survey of twenty members of the Madison Public School's 
Curriculum Department. The survey asked them for their opinion re- 
garding the placement of materials in light of school and societal 
expectations. 

The Passage Size and Preparation 

The actual number of passages used to represent each category 
varied. At times it was three, at other times it was as high as fif- 
teen. Each passage used was randomly selected from the sample of 
passages that had been gathered. 

Passages were about sixty words in length. Each passage 
contained ten blanks. The passage on page seven is an actual passage 
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that was used. As a matter of fact, a11 students at a11 grade levels 
were tested on this passage. For comparison sake, this became an 
Important passage. In the analysis of results. It Is referred to as 
the "comnon passage." 

Using appropriate sampling techniques, each of the fourth 
graders tested responded to 8 of the 144 test passages plus the 
common passage. Each seventh grader tested responded to 6 of the 216 
passages plus the common passage. Ea^h tenth and twelfth grader re- 
sponded to 10 of the 300 passages plus the common passage. 

The Population 

At grade four, 40% of the fourth grade population were 
Included in the assessment. These 1,066 students were selected 
through ji stratified random sampling technique. At grades 7, 10, 
and 12 all students in attendance on the day of the test were In- 
cluded In the assessment. 

PART VI 

QUESTIONS THE ASSESSMENT WILL HELP TO ANSWER 
AND ADDITIONAL QUESTIONS THAT MIGHT BE RAISED 



kmUng^^ axz only KzpiQMZ¥\tativz o(J a ixuigoA body o{^ 
quution^. Vo add yoiUi oml 

Questions the Assessment Will Help to Answer 

' Can students read the textbooks that the Board of Education 
is purchasing for them? 
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- Can students read other school related materials such as 
reference materials and paperback recreational literature? 

- Can students read recreational materials provided for 
them by the public library? 

- Can students read safety messages they are likely to 
encounter, e.g., warnings on packages, warnings on power 
tools, directions for using and dispensing drugs? 

- Are there more girls who are minimally literate than 
there are boys? 

- Are there differences in the percent who are minimally 
literate among the attendance areas across the city? 

- To what extent are factors such as social economic level, 
interest in reading, level of parent education, type of 
school organization, type of school reading program, etc., 
contributing to literacy differences? 

Additional Questions That Might Be Raised 

- Do we have an adequate understanding of the reading process 

- Is it reasonable to expect elementary school child to 
comprehend expository materials which require him to 
understand abstract logical relationships that are outside 
the realm of his personal experiences? 

- Do unsuccessful reading experiences lead to negative 
attitudes toward reading? Do these negative attitudes 
largely prevent the school from doing effective remedial 
work later on? 
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To date, most studies indicate that it is the teacher 
rather than the program that makes a difference In the 
students' acquisition of reading skills. If so, what 
are the teacher skills or teacher strategies that are 
most successful? 

Are there approaches to reading instruction that are 

not now used in Madison that may offer better alternatives 

to producing literate readers? 

Are there teaching skills that teachers should be taught 
to use that can help to prepare a student for his reading 
of a textbook lesson? 

To what extent are instructional packets produced locally 
readable for the students for whom they are intended? 
Is it valid to assurrte that we can move to an individualized 
curriculum if that curriculum is based heavily on written 
materials that many students may be unable to read? 
How readable are the ungraded materials being placed in 
our instructional materials centers and resource centers? 
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APPENDIX A 
DOMAINS AND CATEGORIES 

A. Recreational Literature : Grade Level Used 

1. Magazines (3)^ 4-7-10-12 

2. Stories dealing with family relationships (3) 4-7-10-12 

3. Animal stories (3) 4-7-10-12 

4. Patriotic stories (3) 4-7-10-12 

5. Biographies (3) 4-7-10-12 

6. Adventure stories (3) 4-7-10-12 

7. Student newspapers (3) 4-7-10-12 

8. Mystery stories (3) 4-7-10-12 

B. School : 



1. Student handbook (3) 7-10-12 

2. Paperback list (15) 4-7-10-12 

3. Standardized test instructions (5) 4-7-10-12* 



C. Automobile: 



1. Promotional literature on new cars (3) 10-12* 

2. Auto insurance promotion (3) 10-12* 

3. Automotive license manual (3) 10-12* 

4. Automotive driving tips (3) 10-12* 

5. Operator's and owner's manual (3) 10-12* 

6. Penalty point literature (3) 10-12* 



D. Citizenship : 



1. Wisconsin Constitution (3) 10-12* 

2. Voting directions (3) 12* 

3. Newspapers (40) 7-10-12* 

4. Referenda statements (27) ]H 



E. Leisure-time Activities: 



!• Rule books (sports) (3) 7-10-12* 
2. Directions for assembling toys (3) 7-10-12* 



+ Number indicates the quantity of passages which will represent that 
category. 

* Indicates all grades involved received the same passages. 



f 
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E. Leisure-time Activities : (Continued) Grade Level Used 

3. Recreation department bulletins (3) 4-7-10-12 

4. Directions for games (3) 4-7-10-12* 

5. T.V. Guide (3) 4-7-10-12* 

6. Boy/Girl Scout Manual (3) 4-7-10-12* 

7. Directions accompanying sewing patterns (3) 4-7-10-12* 



F. Occupational : 



1. Vocational School annual ad (3) 10-12* 

2. Instructions on job applications (3) 10-12* 

3. Civil Service test applications (3) 12* 

4. Armed Forces promotional literature (3) 10-12* 

5. Prospective careers promotional literature (3) 10-12* 

6. Instructions for filling out forms (3) 10-12* 

7. School catalogs (3) 10-12* 



G. Reference Materials : 

1. Road maps (3) 7^0-12* 

2. Telephone directory (3) 4-7-10-12* 

3. Encyclopedia (3) 4-7-10-12 

4. Reference books (3) 4-7-1 0-12 



H. Safety : 

}. Fire Department literature (3) 4-7-10-12* 

2. Airplane emergency literature (3) 4-7-10-12* 

3. Civil Defense instructions (3) 4-7-10-12* 

4. Bicycle and pedestrian rules and fire 4-7-10-12* 
evacuation (5) 

5. Warnings on conriercial packaging (10) 4-7-10-12* 

6. Heart, Cancer, and Red Cross literature (3) 7-10-12* 

7. Directions for power tools (3) 7-10-12* 



1. Consumer Material: 



1. Junk mail (3) 7-10-12* 

2. Messages on packages (3) 4-7-10-12* 

3. Recipes (3) 7-10-12* 

4. Catalogs (3) 7-10-12* 

5. Contractual agreements (3) 12* 

6. Banking promotional literature (3) 10-12* 

7. Financial planning literature (3) 10-12* 
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I. Consumer Material : (Continued) Grade Level U 

8. Advertisements (3) 7-10-12* 

9. Conservation/tcology literature (3) 7-10-12* 
10. Consumer magazines (3) 10-1 



J. Textbooks : 

1. Language Arts and Reading (12) 4-7-10-12 

2. Social Studies (12) 4-7-10-12 

3. Science (12) 4-7-10-12 

4. Mathematics (12) 4-7-10-12 

TOTAL - 300 representative passages 



Numbers : 

A. 60 categories in 10 domains. 

B. 300 passages to represent these. 

C. 4th Grade : 18 different booklets of 9 tests each to cover 144 
passages. 

D. 7th Grade : 36 different booklets of 7 tests each to cover 216 
passages. 

E. 10th and 12th Grades : 30 different booklets of 11 tests each to 
cover 300 passages. 
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